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Abstract 

In this paper, we analyse the dependence of the solution of Hamilton- 
Jacobi-Bellman equations on a functional parameter. This sensitivity 
analysis not only has the interest on its own, but also is important 
for the mean field games methodology, namely for solving a coupled 
backward- for ward system. We show that the unique solution of a 
Hamilton- Jacobi-Bellman equation and its spacial gradient are Lip- 
schitz continuous uniformly with respect to a functional parameter. 
In particular, we provide verifiable criteria for the so-called feedback 
regularity condition. 
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1 Introduction 

Sensitivity analysis for systems governed by partial differential equations 
(PDEs) has been a growing interest in recent years. The results of sensi- 
tivity analysis have a wide-range of applications in science and engineering, 
including optimization, parameter estimation, model simplification, optimal 
control, experimental design. Recent progress in sensitivity analysis can be 
found, e.g. in [26J for Burger's equation, [25] for Navier-Stokes equation, 
[H EQl 1211 [22] for elliptic and parabolic equations, [H |13l [IS] for nonlin- 
ear kinetic equations, and references therein. This work contributes to the 
presently ongoing investigation of sensitivity analysis for optimal control 
problems governed by Hamilton-Jacobi-Bellman (HJB) equations. 

We start with a standard stochastic optimal control problem in a finite 
horizon, namely one agent controls her stochastic state evolution to optimise 
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certain objective function within the horizon T > 0. Instead of using a 
stochastic differential equation to describe the evolution, we associate the 
underlying controlled evolution to a family of linear operators, which depend 
on three parameters: time t, control u and a Banach space valued parameter 
11. By applying techniques from operator theory, our framework covers not 
only diffusions, which are considered in most literature, but also a larger 
class of Markov evolutions. 

We assume that the agent can only control her drift, but not the noise. 
For a given parameter curve {fJ-t,t € [0,T]} and certain objective function, 
by dynamical programming principle, the value function satisfies a HJB 
equation. The aim of this work is to study the Lipschitz sensitivity of 
the solution of the HJB equation with respect to the functional parameter 
{fj-t,t G [0,r]}. More specifically, we will show that the unique solution to 
a HJB equation and its spatial gradient are Lipschitz continuous uniformly 
with respect to {fj-t,t G [0)^]} i^i a proper reference topology. 

This sensitivity result has an important application in mean field games 
(MFG), which is a recently developed subject. The MFG methodology was 
developed independently by J.-M. Lasry and P.-L. Lions, see and 
video lectures [12j, and by M. Huang, R.P. Malhame and P. Gaines, see 
[8], [9], ^lOj, pTj. Mean field games methodology aims at describing control 
processes with a large number of agents by studying the limit N ^ oo 
when the contribution of each agent becomes negligible and their interaction 
is performed via certain mean-field characteristics, which can be expressed 
in terms of empirical measures. A characteristic feature of the MFG anal- 
ysis is the study of a coupled system of a backward equation on functions 
(HJB equation) and a forward equation on probability laws (Kolmogorov 
equation). A feedback regularity property of the feedback control is criti- 
cal for solving this system of coupled backward-forward equations. We will 
apply our sensitivity result to mean field games model and give verifiable 
conditions for the so-called feedback regularity condition. 

The organisation of this paper is as follows. Section [2] recalls some basic 
but heavily used concepts in this paper, such as operators, propagators and 
Gateaux derivatives. The main results are presented and proved in Section 
[3l We start by proving the well-posedness of a HJB equation. Then we show 
that, roughly speaking, regularity of a Hamiltonian implies similar regularity 
of the solutions to a HJB equation. In Section HI as an application of this 
sensitivity result, we discuss a mean field games model and give verifiable 
conditions for the feedback regularity property (|4.7p , which was assumed to 
hold and was used as a critical condition (37) in proving Theorem 10 of [8j. 
Section [5] concludes the paper. 
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2 Preliminaries 



In this section, we recall some concepts which are used throughout the paper. 
Let B and D denote some Banach spaces. For a function F : D — >^ B, its 
Gateaux derivative D^F^ji) at ;u G D in the direction % G D is defined as 

if the limit exists. F is said to be Gateaux differentiable at G D if the limit 
exits for all x ^ D. At each point G D, the Gateaux derivative defines a 
function DF{fi) : D ^ B. 

Let i2(D, B) denote the space of linear bounded operators from D to B 
and it is equipped with the usual operator norm || • ||d^.b- In addition, we 
also equip >C(D,B) with the strong operator topology, which is defined as 
the weakest topology such that the mapping A ^ ||^/||b is continuous for 
every / G D as a mapping from £(D, B) to R. Clearly, the strong operator 
topology is weaker than the topology induced by the operator norm. 

For the analysis of time non-homogeneous evolutions, we need the notion 
of a propagator. A family of mappings {[/*'''} from B to B, parametrized 
by the pairs of numbers r <t (resp. t < r) is called a (forward) propagator 
(resp. a backward propagator) in B, if [/*'* is the identity operator in B 
for all t > and the following chain rule, or propagator equation, holds for 
r < s <t (resp. for t < s < r): 

jjt,sjjs,r _ ut,r _ 

Sometimes, the family {U*'^ ,t < r} is also called a two-parameter semigroup. 

A backward propagator {[/*'*', t < r} of bounded linear operators on the 
Banach space B is called strongly continuous if the mappings 

1 1-^ U^'^' for aU t <r, and r i-^- for all t < r, 

are continuous as mappings from R to /^(B,B) in the strong operator 
topology. By the principle of uniform boundedness if {[/*''", t < r} is a 
strongly continuous propagator of bounded linear operators, then the norms 
of {U^'^, t < r} are uniformly bounded for t, r in any compact interval. 

Assume that the Banach space D is a dense subset of B and continuously 
embedded in B. Suppose {U^''^,t < r} is a strongly continuous backward 
propagator of bounded linear operators on a Banach space B with the com- 
mon invariant domain D C B, i.e. if / G D then J/*'*"/ G D for all t < r. 
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Let {Lt,t > 0} be a family of operators Lt G >C(D,B), depending continu- 
ously on t in the strong operator topology. The family {Lt, t > 0} is said to 
generate {[/*''', t < r} on D if, for any / € D, we have 

±U'''f = U'''LJ, ^U''-f = -LM'''f, foralU<s<r. (2.1) 
as as 

The derivatives exist in the norm topology of B and if s = t (resp. s = r) 
they are assumed to be only a right (resp. left) derivative. 

One often needs to estimate the difference of two propagators when the 
difference of their generators is available. To this end, we shall often use the 
following rather standard trick. 

Proposition 2.1. For i = 1,2 let {L\,t > 0} &e a family of operators 
L\ G £(D,B), depending continuously on t in the strong operator topology, 
which generates a backward propagator {U^'^ ,t < r} in B satisfying 

ai := sup max ] ||C/f''"||B^B, ^2'' ||b^b [ < 00. 

t<r J 

//D is invariant under {Ui^,t < r} and 

a2 := sup |!C/{'''||d->d < 00, 

t<r 

then, for each t < r, 

U'/ - U'/ = U'/iLl - LDurds (2.2) 

and 

111/2'' - Ul''' Wu^B < aia2{r - t) sup ||L^ - ||d^b- (2.3) 

t<s<r 

Proof. Define an operator-valued function Y{s) := U2^U^'^. Since ?7*'*, 
z = 1,2, are identity operators, so Y{r) = 1/2^' and Y{t) = U^''' . By (|2.ip . 
we get 

t/f - U'/ = U'2''ur\:=t = ^ {U'2''ur) ds 
= jy'2'Llur-U'2'Llurds 
= jy'^\Ll-L\)Urds, 
which implies both ([22]) and (jO]) . □ 
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3 Main results 



Let C := Coo(R'^) be the Banach space of bounded continuous functions / : 
R*^ — >■ R with \imx^oof{x) = 0, equipped with norm ||/||c ■= sup^. |/(a;)|. 
We shall denote by := C^(R'^) the Banach space of continuously dif- 
fcrcntiablc and bounded functions / : R*^ R such that the derivative /' 
belongs to C, equipped with the norm ||/||ci •= sup^, |/(a;)| + sup^, \ f'{x)\, 
and by := C^(R'^) the Banach space of twice continuously differen- 
tiable and bounded functions / : R*^ R such that the first derivative 
/' and the second derivative /" belong to C, equipped with the norm 
||/||c2 := sup,|/(x)| +supj/'(a;)| +supjr(x)|. Let Cup := Cupi^.'') 
denote the space of Lipschitz continuous functions / : R'' — >■ R, equipped 
with the norm ||/||c,,, := sup, |/(x)| + sup,,, ^^^g^. 

Note, that = C^(R'^) is a Banach space which is densely and contin- 
uously embedded in C = Coo(R'^). Depending on the modelling assumption, 
the Banach space can be replaced by other examples of functions spaces, 
such as the space of Holder continuous functions, and our methods can be 
applied in a similar way. 

Let T > be fixed and ^ be a subset of a Euclidean space, interpreted 
as the set of admissible controls, with the Euclidean norm | • |. Take ^A to be 
a bounded, convex, closed subset of another Banach space S, equipped with 
the norm || • ||s. In applications, very often the Banach space S is taken as 
the dual space (C^)* of and the set Ai is taken as the set of probability 
measures on R'^, which is denoted by V(R'^). 

Let 

{A[t,n,u] -.t e[0,T],ij, e M,u eU} (3.1) 

be a family of bounded linear operators A[t,iJ,,u] : — )■ C, defined on 

the same domain for all (t, ^u, u) G [0, T] x x U. For each (t, fi, u) € 
[0, T] X xZ^, the linear operator A\t, /x, ii] : i->- C is assumed to generate 
a Feller process with values in R'^ and to be of the form 

A[t, M, u]f{z) = {hit, z, M, u), Vf{z)) + L[t, fi]f{z), (3.2) 

where the coefficient h : [0, T] x R*^ x A4 x U ^ R*^ is a vector-valued 
function. For each pair (i,/x) € [0,T] x M, the linear bounded operator 
L[t,ij] : — >■ C is of Levy-Khintchine form with variable coefficients: 

L[t, fi]f{z) = l{G{t, z, /x)V, V)f{z) + {b{t, z, m), V/(z)) 

(3.3) 

+ / {f{z + y)-fiz)-iVfiz),y)lBM>it,z,^i,dy), 
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where V denotes the gradient operator and 1^^ denotes the indicator func- 
tion of the unit bah in R'^; for each {t, z, fi) G [0, T] x R"' x A^, G{t, z, /x) is 
a symmetric non-negative matrix, b{t, z, fi) is a vector, z, fj,, •) is a Levy 
measure on R'^, i.e. 

/ nhn(l, |yp)z/(t,z,/i,(iy) < oo, z, /i, {0}) = 0. (3.4) 

Jr.'' 

We assume that the mappings {t,z,fi) — )• G{t,z,fj,), {t,z,iJ,) b{t,z,fi) 
and {t,z,fj,) v^t, z, fj,, ■) are Borel measurable with respect to the Borel 
cj-algebra in [0,T] x R^ x A^. 

Let (^xj-^'^'^^'^ : t G [0, T]) be a controlled stochastic process on a prob- 
ability space (n, V) with values in R"' and generated by the family of 
operators {A[t,^,u\ : t G [0, T], /i G Al, n G U} in of the form dO]) . 

The control process is described by a stochastic process {u.} = {ut &U : t G 
[0,T]}. Let C{[0,T], ^A) be a space of continuous curves {/u.} = {nt G M, 
t G [0,T]}. For notational brevity, in the following we write (Xf : t G [0,T]) 
instead of (x/''-^'^"-^ : t G [0,r]). 

Given a curve {fi.} G (7([0,T],7W), we aim to maximize the expected 
total payoff 



E 



J{s,Xs,Hs,Us)ds + V'^{XT,^J-T) 



J{s,Xs,Hs,Us)ds + V'^{XT,fiT) 



over a suitable class of controls {ut G U,t G [0,T]} with a running cost 
function J : [0, T] x R'^ xMxU and a terminal cost function V'^ : R*^ x 
M^n. Therefore, for a given curve € M : t e [0,T]} G C{[0,T],M), 
the value function V : [0, T] x R'^ R starting at time t and state x is 
defined by 

V{t,x; {fis ■ s e [t,T]}) := supE^; 

{u.} 

(3.5) 

By standard arguments from dynamic programming principle and assuming 
appropriate regularity, the value function V satisfies the Hamilton-Jacobi- 
Bellman (HJB) equation 

dV 

- — (i, x; {fi.}) = Ht{x, VF(t, x; {^.}), ^t) + L% iit]V{t, x; {fi.}) 
V{T,x;{^i.})=V^{x;fiT), 
where the Hamiltonian i7 : [0, T] x R'' x R^ x 7W R is defined by 

Ht{x,p, n) = max(/i(t, x, fi, u)p + J{t, x, fi, u)). (3-7) 
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Our main aim of this paper is to investigate the sensitivity of the solu- 
tion V{t,x; {n-}) of the HJB equation (j3.6p with respect to the functional 
parameter {fi.} € C{[0,T], M.). In the first place, we need to show that for 
each fixed curve {//.} € C{[0,T], ^A), the HJB equation p.6p is well posed. 
Then, we discuss the sensitivity of the solution to (|3.6p with respect to the 
parameter {/i.} € C([0, T],7W). In fact, we shall show that the unique so- 
lution V and its spatial gradient are Lipschitz continuous uniformly with 
respect to {^.}. 

3.1 Main assumptions 

For any fi G M, define the set ^A — fi := {rj — fi : r] £ M}, which, as a subset 
of S, is equipped with the norm || • ||s- In the analysis below, we need the 
following assumptions: 

(Al): the Hamiltonian H : [0, T] x R'^ x R'^ x R is continuous 

in t and Lipschitz continuous uniformly in x on bounded subsets of p. Fur- 
thermore, it is Lipschitz continuous uniformly in p, that is there exists a 
constant ci such that for all x G R'^, G A4 and t € [0,T] we have 

\Ht{x,p,fi)-Ht{x,p',fi)\<ci\p-p'\ forp,p'GR'^. (3.8) 

It is bounded in p = 0, that is there exists a constant C2 > such that 

\Ht{x,0,fi)\ < C2 for all x gW^, fi £ M,t e[0,T]. (3.9) 

For each t € [0, T] and x,p € R'^, the function fj, >->■ Ht{x,p, fi) is Gateaux dif- 
ferentiable in any direction x G — /^i such that (t, x,p, //) i-7> D^Ht{x,p, fi) 
is continuous and satisfies that for each bounded set B C R*^ there exists a 
constant C3 > such that 

sup\D^Ht{x,p,fi)\ <C3\\x\\s for allt £[0,T],x e'R'^,n e M. (3.10) 

(A2): the mapping 

[0,T] X M ^ C{C'^,C), {t,fi) ^ L[t,fi] 

is continuous in the strong operator topology. For any {fi.} £ C{[0,T], A4), 
the operator curve {L[t,fj,t] '■ t E [0, T]} generates a strongly continuous 
backward propagator < s} of operators S >C(C,C) with the 

common invariant domains C'^ and C^. There exists a constant C4 > such 
that for all < t < s < T we have 

maxjllC/Jj^^^^llc^c, Ilci^ci, WUlllyW&^c^} < C4- (3.11) 
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The propagator has a smoothing property, that is for each 0<t<s<Twe 
have 

f/J;^ ■.C^C\ C/f^ j : Cup ^ C\ (3.12) 
and there exists a /3 € (0, 1) and constants C5, cg > such that 

||^g}0||ci < c,{s - tr^WHc, rf^iV'llc^ < ceis - t)-^\mc,,^ 

(3.13) 

for all G C and ^ € Cup. 

(ii) for any t G [0, T], the mapping fi L[t, fi] is Gateaux differentiable in 
any direction x € M — ^jl, such that the mapping ^ ^ D^L[t, is continuous 
in the strong operator topology of >C(C^, C). There exists a constant C7 > 
such that for each /i G and x £ ~ we have 

||Z)xL[t,^]||c2^c ^crllxlls foraht G [0,r]; (3.14) 

(A3): for any ^ G M., the mapping x i-)- ^^(x; ^u) is twice continuously 
differentiable, and for each x G R'^ the mapping ^ ^ V'^ {x; /i) is Gateaux 
differentiable in any direction x S ~ such that the mapping (x, ^) 1-^ 
D^V'^ {x] ij) is continuous. There exists a constant cg > such that 

Px^''(-;/^)llci <C8||xl|s. (3.15) 

Remark 3.1. If the Banach space S is given as the Euclidean space R, then 
corresponds to the standard partial derivatives and are denoted by d/da 
for a G R. 

The smoothing conditions (|3.12p and (j3.13p in assumption (A2) are es- 
sential and critical in the following analysis. Let us show two basic examples 
which satisfy assumption (A2): the diffusion operator 

L[t, fi]f{x) = ^{a\t, x, m)V, V)/(x) + {b{t, X, ^), V/)(x) (3.16) 

with smooth enough functions b, a, see e.g. in |23j and references therein. 
The operators {L[t,^],t G [0,T]} generate the stochastic process {X{t),t G 
[0,T]) which obeys the stochastic differential equation 

dXt = b{t, Xt,fxt)dt + a{t, Xt,Ht)dWt, 

where is a standard Brownian motion. 

Another example is given by stable-like processes with the generating 
family 

L[t,f,]f{x) = a(t,x)|Ar(-)/2 + {bit,x,fi),Vf){x) (3.17) 
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with smooth enough functions a, a such that the range of a is a compact 
interval of positive numbers and the range of a is a compact subinterval of 
(1,2). In both cases, each operator Uj^^ , t < s, has a kernel, e.g. it is given 
by 

U'/f(.x) = I G^{t,s,x,y)f{y)dy (3.18) 
with a certain Green's function G^, such that for every x € R'^ and t < s, 
sup [ \V^Gf,{t,s,x,y)\dy <c{s-t)-^ (3.19) 

for a constant c > 0. Here, in the first case ()3.16p . we have /3 = ^ and in the 
second case (I3.17j) . we have /3 = (inf^^ a{x))~^ . In both cases the smoothing 
conditions (|3.12p and (|3.13p are satisfied, see [H] and references therein. 

3.2 Well-posedness of HJB equation 

In this subsection, we prove the well-posedness of the HJB equation (13. 6p . 
For this purpose, we can fix {nt € M. : t & [0,T]} G C{[0,T], A4) and thus, 
we omit the dependence of the functions H, L, on the parameter /i G 
and we consider the Cauchy problem 

dV 

-^(t, x) = Htix, Wit, x)) + LtV{t, x) 
V{T,x) = V^{x) 
with the Hamiltonian H : [0, T] x R'^ x R'^ — )■ R defined by 

Ht{x,p) = max(/i(t, x, u)p + J{t, x, u)). (3.21) 

and the operator Lt : C for each t G [0,T]. By Duhamel's principle, 

if F is a classical solution of (|3.20p . then V is also a mild solution of (j3.20p . 
i.e. it satisfies 

Vit, x) = (?7*'^y^(-))(x) + U''-'Hs{ ■ , VV{s, ■)){x)ds (3.22) 

for ah t G [0, T] and x G R'^. 

For the sensitivity analysis of this work, it is sufficient to consider only a 
mild solution, which exists under weaker conditions than a classical solution. 
For this reason, we will establish the existence of a unique mild solution. In 
this subsection, we mostly follow Chapter 7 in [Tl], where one also can 
find details for existence of a classical solution. We present this result for 
completeness on the level of generality which is required by what follows. 
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Theorem 3.1. Assume conditions (Al) and (A2). // the terminal data 
V'^{-) is in C^, then there exists a unique mild solution V of ()3.20p . satis- 
fying V{t, •) G for all t G [0,r]. 

Proof Let C^j,([0,T],Ci) be the set of functions </>: [0, T] x R'^ ^ R, which 
satisfy ^(T,x) = V'^{x) for ah x G R'^, (/>(t,-) G for each t G [0,T] and 
the mapping t (t>{t,-) is continuous as a mapping from [0,r] to C. We 
equip this space with the norm 

ll'^llc^^([o,Ti ci) sup ||0(t,-)||ci- 

te[o,T] 

Note this definition of the set C^-r([0, T], C-^) is not standard in the sense 
that it the continuity is considered from [0,T] to C, but not from [0,T] to 
C\ 

Define an operator ^ acting on C^y([0, T\,C^) by 

x) := (C/*'^y^(-))(x) + U'^'Hsi-, V0(s, ■))ix)ds. (3.23) 

Clearly, the mapping t — )• •) is continuous since the propagator C/*'-^ 

is strongly continuous in t and the integral term is continuous in t. 

Since V'^{-) G and the family {C/*'^,0 < t < T} is bounded as 
a family of mappings from to C^, we have U^''^V'^{-) G and it is 
uniformly bounded on < t < T. By the triangle inequality and (I3.8p . (l3.9p . 
for each t £ [0,T] 

\\Ht{;V^{t,-mc < \\Ht{;0)\\c + \\Ht{;V^{t,-))-Ht{; 0)\\c 

< C2 + Ci||V0(t,-)||c 

< C2 + cimt,-)\\ci. (3.24) 

The smoothing condition (I3.12p guarantees that U*''^Hs{-, S/(p{s, •)) G for 
each 0<t < s <T. The conditions ([3TT]1 . ([3T3]1 and the inequality ([3:21) 
imply for each t G [0, T] that 

\\^mtr)\\c^ <\\U'^^V''i-)\\ci+ £ \\U'''H,{;Wcf>{s,-mc^ ds 

<c4\V^{-)\\c^ + C5 j\s - t)-f^\\H,{-,V(l)is,-mc ds 

<C4||y^(-)||ci+C5 C2 + C1 sup \\4>{s,-)\\ci 
V t<s<T 



1-/3 
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It follows that the operator maps C^t([0, T], C^) to itself, i.e. 

Conditions <^ and (imjl imply for every £ C^r{[0,T],C^) and 

te[0,T] that 

\\^{c^'){t,.)-^{cp'){t,-)\\c^ 

||[/*'^[/7,(-,V0i(s,-))-^s(-,V02(s,-))]||ci ds 

< c^{s - t)~^ci\\V4>\s, ■) - V</)2(s, .)\\c ds 

<ciC5 ^^~^^' ^ sup \\ct>\s,-)-ct>\s,-)\\c^. (3.25) 

J- — P t<s<T 

By choosing to small enough, it follows that the mapping ^ is a contraction 
in CyT([0, r], C^) for time t € \T — tQ,T]. Consequently, by the contraction 
mapping principle, ^ has a unique fixed point for t € [T — to,T]. The well- 
posedness on the whole interval [0, T] is proved, as usual, by iterations. □ 

By the wellposedness of equation p.22p , its solution defines a propagator 
in C^. Standard arguments, see e.g. [3], show that this solution is a viscos- 
ity solution to the original equation p.20p and it solves the corresponding 
optimization problem. 

3.3 Sensitivity analysis of HJB 

In this subsection, we analyse the dependency of the solution of the HJB 
equation (13. 6p on the functional parameter {/i.} G C{\Q,T], M). Under the 
conditions (Al) and (A2), Theorem l3.1| guarantees the existence of a unique 
mild solution •;{//.}) of (|3.6p for each fixed curve {/i.} E C([0,r],A^). 
The following observation plays an important role in this work. Let 
be in C([0,T],7W) and let a G [0,1]. Since M. is convex, the 

curve 

{/i^} + a{(^2 _ ^1) I _ ^ ^(^2 _ ^1)^^ ^ 
belongs to C([0, r],A^). Thus, we can define the function 

: [0,r] xR'^^R, V^{t,x) ■.= V{t,x;{^i^} + a{{i? - (3.26) 
and have the relation 

V(t,x;{/i2}) - y(t,x;{^i}) = Vi{t,x)-V^{t,x). (3.27) 



11 



Furthermore, for € C{[0,T],M) define 



(3.28) 



max(/i(t, X, /ii + a(/ij - fil),u)p + J{t,x, fi} + a{^4 - /it),n)) 



Vjix) :=y^(x;/ii. + a(/i2 -4)) 



(3.29) 
(3.30) 



with a G [0,1], t G [0,r] and (x,p) G R'^ x R'^. Then the sensitivity 
analysis of the solution of ()3.6p with respect to a function parameter {/i.} G 
C([0,T],A^) can be reduced to the one of the solution to the following 
Cauchy problem with respect to a real parameter a G [0, 1]: 



VaiT,x) 



-Ho,4x,VVUt,x)) - Lo,[t]Vo,it,x) 
Vlix). 



(3.31) 



The sensitivity analysis with respect to a G [0, 1] consists of two steps. 
First, we omit the Hamiltonian term in (I3.3ip and only consider the sen- 



sitivity of the evolution Va{t,-) = Ua^Vj{-), where for each a G [0,1] 



the propagator : t < s} is generated by the family of operators 

{Lo[*] : t G [0)^]}- This intermediate step gives us some interesting re- 
sults. Then we take into account the Hamiltonian term and complete the 
analysis. 

Theorem 3.2. Assume conditions (A2),(A3) and define 



W : [0, 1] X [0, T] X R'' 



R-^ 



W^{t,x) = U'fv;^{x). 



(3.32) 



Then for each t G [0, T] and x G R'^, the mapping a — >■ Wa{t, x) is Lipschitz 
continuous with uniformly bounded Lipschitz constants, more precisely for 
every ai, 02 G [0, 1] with ai ^ 02, there exists a constant c > such that 

\\Wa^{tr)-Wa,{t,-)\\& 



\ai — 02 



< c 



sup 

7e[Q!i,a2] 



da 



+ {T-tY~ 



sup 

7e[oi,02] 



da 



for every t G [0, T] . 
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Proof. From (|3.32p . for each 01,02 G [0, 1], t G [0,T] and x S R'^, we have 
U!:^ [Vl - (x) + f C/i^ - ) Vl (x). (3.33) 



By the condition (A3), for each x G R'^ the mapping a — )• Vj{x) is 
differentiable and the derivative -g^{-) belongs to C^. Since for any < 
t < T and oi G [0, 1], C/^f : ^ C\ together with (ISTTT) we have 



(V^ 



vl,) (Oil 



u, 



t,T 



"1 dVJ' 

' (0^7 



a2 



da 



< C4|ai — 02 1 sup 

7£[ai,a2] 



Cl 



da 



(3.34) 



By (12. 2p in Proposition 12.11 and the smoothing property (I3.13p . we have 
U'a^ - = C/*'f (L,, W - L,, [s])K^^ds ■.c^^c\ 

Together with the condition A2 (ii) that for each t G [0, T] the mapping 
a 1-^ La[t\ is differentiable and ^^[t] : — > C, we have 

\Ujjt,T _ rjt,T\ yTtM\ 
II \'~' ai "^02 / '^02V )\\ci 

f-T 

<CiC5 {s-t)-'^ds sup lli^ajs] - i^a2[s]||c2^cl|V;^(-)llc2 

Jt selt.T] 



< C4C5 — — \ai - 02! sup 

7G[ai,a2] 



da 



l^a^(-JIIC2- 



(3.35) 



Therefore, from ()3.33p together with ()3.34p and (I3.35p . we complete the 
proof. □ 
In this work, we are only concerned with the Lipschitz continuity of 
the solution of the HJB with respect to the parameter. It is interesting to 
know whether the mapping a 1— )• Wa{t, •) is differentiable for each t G [0,T]. 
For the completeness, the next proposition will show the existence of the 
derivative ^^(^r) iii C and present its' explicit expression. 

Proposition 3.1. Assume conditions (A2) and (A3). Then 
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(i) for each < t < s < T , the mapping a i— )■ Ua'^ is differentiable in the 
sense of the strong operator topology in £(C^,C) and the derivative 

r\jjt,S 

^° has the representation 

Tt,S 



da 



U'f^[r]U'^' dr, forO<t<s< T. (3.36) 



(ii) for each t E [0,T], the mapping a i— )• Wa{t,-) defined in (j3.32p is 
differentiable as a function from [0, 1] to C and the partial derivative 
^^(t,-) can be represented by 

Proof, (i) Since the operator La[t] is differentiable in a for each t € [0,T], 
together with (j2.2p in Proposition 2.1, for ai,a € [0,1] with ai ^ a, we 
have 



ai — a a\ — a 



X da ^ J " 



with some 7 G [ai,a]. By (12. 3p in Proposition 12.11 for each < t < r < T 
the mapping a i-> C/q'^ is continuous as a function from [0, 1] to £(C^, C) 
in the strong operator topology. Then by standard density arguments, for 
each < t < r < T, the mapping a 1— >■ Ud is also continuous as a function 
from [0, 1] to C{C, C) in the strong operator topology. Together with the 
smoothing property (|3.12|) and the condition that for each r € [0, T] and 
7 € [0, 1], ^^[r] : — )• C, we have that the derivative 

— — := hm 

oa Qi-i-a ai — a 

exists in the strong topology in £(C^, C) and equals to 



da 



(ii) By condition (A3), the mapping a i-> Vj{-) is differentiable and 
the derivative exists in C , namely, limQ,^_^Q, _ " exists and belongs to 
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C^. Since the mapping a i— )■ JJ^a is continuous as a function from [0, 1] to 
£(C, C) in the strong operator topology, hence for any t € [0,T], 



hm 



"1 



a 



Finally, from (j3.32p , the derivative 



da 



Wc,,it,-)-Wait,-) 




+ lim 



Tjt,T 



Tjt.T 



a 



da da 
exists in C for each t € [0, T] and any a G [0, 1]. 



□ 



Remark 3.2. If one would have that, for each <t < s <T, the mapping 
a i-> Ua^ is continuous from [0,1] to C{C,C^), then lima^^Q C/o** exists as 
an operator from C to . Then one would have that the mapping a i-^ 
Wa{t, •) is differentiable as a function from [0, 1] to C^. 

Theorem 3.3. Assume the conditions (AI), (A2) and (A3). Then we 
have the following: 

(a) For any T > 0, the mild solution Va of (I3.3ip is Lipschitz continuous 
with respect to a i.e. there exists a constant c = c{T) > such that 
for each ai,Q2 G [0, 1] with ai ^ 02, 



sup 

te[o,T] 



\VaAtr)-Va2{t,' 

\ai — 02! 



sup 

7G[Q!i,a2] 



da 



+ sup 

ci {*.p)e0 

7e[Q!l,Q:2] 



da 



(3.38) 



+ sup 

te[o,T] 

7e[ai,a2] 



da 



[t] 



C2^C 



where = {it,p): te[0,T],\p\< snpt^[o^T]\\Va{t,-)\\c^}- 
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(b) The mild solution V of ()3.20p and its spacial derivative W are Lip- 
schitz continuous uniformly with respect to {fJ..}, that is, for each 
G C{[0,T], M), there exists a constant k>0 such that 

sup \\V{t,-;{l^^})-V{t,-,{fi^})\\ci <k sup yl-fiUs (3.39) 

ie[0,T] t£[0,T] 

and 

sup \\VV{t,.;{^'})-VV{t,.;{^^})\\c<k sup Wfij-fi^s- (3.40) 

t€[0,T] te[0,T] 

Proof, (i) Recall in the proof of Theorem 13.11 for any a € [0, 1], the unique 
solution Va is the unique fixed point of the mapping 

cP ^ ^aicp), C^t([0,T], C^) ^ C^t([0,T], C^) 
defined by, for each t G [0, T] 

^aim, •) = U'fvJi-) + U'^'H^A-, V0(s, ■))ds. (3.41) 

For any ai G [0, 1], « = 1, 2, let Va^ be the unique fixed point of the mapping 
*a,, i.e. 

^a, =^'a.(K.J, fori = 1,2. 

Then from ()3.4ip we have 

Vc,,{t,-)-Va,(,t,-)=^aAVaAt,-))-^aM2(.t,-)) 



+ r U'f^H^,,s{-,VV^Asr))ds- r U'f^H^,A-,VV^,{s,-))ds 
Jt Jt 

+ {U'f, - HaU; VK., is, ■))ds 

+ £ U'f^ (i?„„,(-, vy„,(s, •)) - i/a2,.(-, vy,,(s, •))) ds 

+ f/*'^ {H^,A;VV^,{s, •)) - Fa2,.(-, V14,(s, •))) ds 
=: A1+A2 + A3 + A4. (3.42) 
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By theorem 13.21 there exists a constant c > such that 



< c|qi — sup 

7g[oi,Q!2] 



da 



ci 



c\ai — a2\{T — t)^ ^ sup 



se[t.T] 

7g[ai,a2] 



da 



\\Vn-)\\c^. (3.43) 



By proposition 12.11 and the differentiabihty of La[t\ in a for each t G [0,T], 
we have 



||A. 



2||Cl 



t \Jt 



Ci 



r rs 



< \ai - a2|c5C6 




(r - O'^^Cs - r) 



t Jt 



< \ai - a2\c^CQm / {s - t) 



sup 

re[t,s] 



9a 



i^ai,s(-,VV;,(s,-))||c^,,ds 



da 



ds 



< \ai - a2\c5CQm 



1 



2-2/3 



(T-t) 



2-2/3 



sup 

•e[t,T] 



da 



(3.44) 



with a € [0,1] and m := sup(g p)gQ ||ffa^s(-,p) Hc^ip < 00 where Q = 
{{s,p) : s € [0,T], \p\ < supsg[o^r] ||K.(s, Ollci}- % the condition (Al), for 
each t G [0,T] and x,p G R*^ the mapping a Ha,six,p) is differentiable 
and the derivative is continuous, we have 



i|A; 



3llci 



d J-f 

U'^^{ar-a2)-^{;VV^Asr))ds 



< \ai - "2I ^ C5(s - ty 



dHa 



da 



-(•,V14,(s,-)) 



ds 



<|ai-a2|c5 — — sup 

J- - P (s,p)G0 



dH, 



da 



,P) 



(3.45) 



where = {(s,p) : s G [0,r],|p| < sup,e[o,T] ll^ai(s, Ollci}- By and 
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([3:8]) . we get 



||A, 



U'f^ {H^,,s{;VVa, (s, ■)) - H^U; («, •))) ds 
< ciC5 j\s - t)-^\\VV^, {s, •) - VVo^.is, •) lie ds 



<CiC5 / is-t)-^\\Vo,,is,-)-Vo,,is, 



1 



Ici 



ds 



^^1^5:^ ^{T-t)^-^ sup ||K.,(s,-)-K.2(s,-)llci- (3.46) 

It follows, from (l3:i2]l together with the estimates (fSliD . (fOSi) 

and ([336]), that 



sup ||K*i(i, •) - K.2(*'-)llci 

t€[0,T] 



(3.47) 



< cjai — 02 1 sup 
7e [01,02] 



da 



+ c\ai — a2\{T — t)^ ^ sup 



ci 



tG[0,T] 
76(01, 02] 



9a 



1 



sup 



2 — 2/3 te[o,T] 

7S[oi,02] 



C2->C 



llK.^(-)llc^ 



^7 



5a 



C2^C 



2-1-/3 

+ |ai-a2|c5:; ^ sup 

^ ~ P {t,p)e0 
7e[Qi,02] 



7,t 



C1C5- ^T^-/^ sup ||yoi(t,-)-^^02(t,-)llci- 

^ - P tG[0,T] 



(3.48) 



/3-1 



For ciC5j^T'^->^ < 1, i.e. T < , we have inequality (13381) . For 

any finite T > 0, the proof follows by iterations. 

(ii) By the definitions of L„[t], Ha,t{x,p), V^{x) in ([3:28]) . (15:29]) . ([330]) 
respectively and the assumptions (13.10p . (|3.14p . (I3.15p . for any {fJ-^}, {fJ-^} € 
C([0,T],7W), the statement follows from the equation (I3.27P and the in- 
equality p.38p by setting qi = 1 and 02 = 0- □ 
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4 Application to mean field games 



In this section, we apply the sensitivity results in Theorem 13.31 to a mean 
field games model and give verifiable conditions for the so-called feedback 
regularity condition. 

Let us consider a continuous time dynamic game with a continuum of 
players and a terminal time T > 0. Take S = (C^)*, as the dual Banach 
space of C^, and Ai = V(R'^), as the set of probability measures on R'^. 
In this game, all players are identical so it is symmetric with respect to 
permutation of the players. Choose one of the players and call it the reference 
player. We use the controlled stochastic process {Xt : t G [0,T]) to model 
the controlled state dynamics of the reference player. At each time t G 
[0, T], the reference player knows only his own position Xt and the empirical 
distribution of all players fit G 'P(R'^). 

Remark 4.1. For a better understanding of the stochastic process {Xt : 
t G [0,T]), one may think that the controlled dynamics is described by a 
stochastic differential equation. For a very particular case of our model, set 
L[t,ij] = ^(o"V,V) with a constant a, i.e. the operator L[t,fi] generates a 
Brownian motion {aWt : t > 0}. Then one can write a stochastic differential 
equation corresponding to the generator (j3.2p as 

dXt = h{t, Xt,fit, ut) dt + adWt for all t>0. 

In fact, this is exactly the case which was considered in the initial work 
on the mean field games Q 0, 0, \18^ . In our framework, this controlled 
dynamics of each player is extended to an arbitrary Markov process with a 
generator ()3.2p depending on a probability measures fi. 

The empirical distribution evolution of all players in the state space R'^, 
denoted by {pLt G 'P(R'^) : t G [0,T]}, is described by the evolution equation 

/ g{y) ixtidy) = ( {A[t,fit,ut]g{y)) fitidy) for all 5 G (4.1) 
dt J^d J^d 

with a given initial value fiQ G 'P(R'^). The equation (14. ip is a controlled 
version of a general kinetic equation in weak form and very often written in 
the compact form 

-^{9, l^t) = {A[t, Ht,ut]g, Ut). (4.2) 

See [151116] for more discussion on equation (j4.2p and its well posedness with 
open-loop controls under rather general technical assumptions. A solution 
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{^t ■ t £ [0, T]} of equation ()4.2p is called the (probability) measure flow. Let 
C^(j([0, T], T'(R°')) be the set of continuous functions t ^ with G 'P(R'^) 
for each t E [0, T] and with the norm 

||^||(C2). := sup \{g,^j)\ = sup 

Il9llc2<l Ilsllc2<l 

In this game, the reference player faces an optimisation problem de- 
scribed by the HJB equation (I3.6p . If the max is achieved only at one point, 
i.e. for any {t,x,fi,p) € [0,r] x R'^ x 'P{'R'^) x R'^, 

arg max(/i(t, x, /i, u)p + J(t, x, fi, u)) 

is a singleton, then one can derive the unique optimal control strategy from 
the solution of ([32]). For any given curve {/Xt : t G [0,r]} G C^o([0,r],P(R'^)), 
let the resulting unique optimal control strategy be denoted by 

u{t,x;{^is ■■ s £[t,T]}) for aht G [0,r],x G R'^. (4.4) 

Substituting the feedback control strategy (14. 4p into (I4.2p yields the closed- 
loop evolution equation for the distributions fit 

^(5,Mt) = {A[t,iit,u{t,x]{iXs ■■ s G [t,T]})]g,fit). 

The mean field game methodology amounts to find an optimal control 
strategy {u,} for each agent and a measure fiow {/i.} such that the following 
two coupled equations 

^{g, ^t) = {^[t, fJ't,u{t, x; {fis ■■ s € [t, T]})]g, fit) (4.5) 



g{x)fi{dx) 



(4.3) 



and 



dV 

- — (t,x) = max{h{t,x,fj.t,ut)W + J{t,x, fit,ut)) + L[t, ij.t]V{t,x) (4.6) 

Ot u.&A 



V{Tr;fiT) = V'{-,fiT) 

hold. The control strategy u applied in (|4.5p is derived from the HJB equa- 
tion (|4.6p . Since the controlled kinetic equation (j4.5p is forward and the 
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HJB equation ()4.6p is backward, this system of coupled equations is referred 
to as a coupled backward-forward system. 

To solve this coupled backward- forward system (j4.5p - (|4.6p . it is critical 
that the resulting control mapping u (|4.4p satisfies the so-called feedback 
regularity condition (see e.g. [8j), i.e. for any {r]t : t € [0,r]}, {.^t : t € 
[0,r]} € C^o([0,r],P(R'^)), there exists a constant ki > such that 

sup \u{t,x; {7]s : s e [t,T]}) - u{t,x] {Cs ■ s £ [t,T]})\ 

{t,x)G[0,T]xRd ^^^^ 

< ki sup \\7]s - 6ll(C2)*- 
se[0,T] 

Theorem 4.1. Suppose L, H and in (|3.6p - (|3.7p satisfy the conditions 
(Al), (A2), (A3) respectively. Assume additionally the max in (|3.7p is 
achieved only at one point, i.e. for any (t, x, fi,p) S [0, T] x R*^ x ViTV^) x R'^ 

argmax(/i(t, x, fj,, u)p + J{t, x, fi, u)) (4.8) 

is a singleton and the resulting control as a function of {t,x, fj,,p) is contin- 
uous in t £ [0,T] and Lipschitz continuous in {x,fi,p) G R"^ x V(R.'^) x R*^ 
uniformly with respect to t, x, fi and bounded p. Then, given a trajectory 
{/i.} G C^o([0,r],P(R'^)), the feedback control mapping 

u{t,x;{ns : s G [t,T]}) 

defined via equations ()3.6p - (|3.7p . is Lipschitz continuous uniformly in {/i.}, 
i.e. for every {r,.}, {C} G ([0, T], P(R'^)), 

sup \u{t,x;{r]s : s G [t,T]}) -il{t,x;{Cs : s G [t,T]})\ 

{t,x)G[0,T]xRd ^^g^ 

<ki sup Wvs - Cs\\(c2)* 
se[o,T] 

with some constant ki > 0. 

Proof. By Theorem 13. 3| together with the assumption that the resulting 
unique control mapping is Lipschitz continuous in (x,/i,p), we conclude 
that the unique point of maximum in the expression 

max{/i(t, X, iit-,u)W{t, x; {/i.}) + J{t, x, Ht,u)} 

has the claimed properties. □ 
We give two examples where the uniqueness condition (14. Sp holds. 
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Example 4.1 (//oo-optimal control, see [23] for its systematic presentation). 

Assume that the running cost function J is quadratic in u, i.e. 

J{t, X, fi, u) = a{t, X, n) — 9{t, X, ia)u^ 

and the drift coefficient h is linear in u, i.e. 

h{t, X, fj,, u) = f3{t, X, fi)u, 

where the functions a, 13,6 : [0, T] x R'^ x 7'(R'^) — )■ R and 6{t, x,fi)>0 for 
any Thus, we are maximising a quadratic function over control u. 

It is easy to get an explicit formula of the unique point of maximum, i.e. 

u = -^{t,x,n)p. 

Thus, the HJB equation (j3.6p rewrites as 
dV 

— (t, x; {^i.})+^{t, X, fit){VVf{t, x; {^l.})+a{t, x, //t)+L[t, fit]V{t, x; {fi.}) 
which is a generalized backward Burger's equation. 

Example 4.2. Assume h{t, x, fJ,,u) = u and J{t, x, fi, u) is a strictly concave 
smooth function ofu. Then Hf is the Legendre transform of —J as a function 
of u, and the unique point of maximum in (j3.7p is 

. dHt 



dp 

If J{t,x, iJ,,u) has the decomposition 

J{t, X, /X, u) = V{x, fi) + J(x, u) 

for F : R'^ X P(R"') ^ K, J : K"^ xU H and L{t, fi) = A, the correspond- 
ing coupled backward-forward system (|4.5p - (|4.6p turns to system (2) of flSf 
( only there the kinetic equation is written in the strong form and in reverse 
time). 

Remark 4.2. Let us stress again that in (j3.39p . (j3.40p the space S is an 
abstract Banach space, but in application to control depending on empirical 
measures, we have in mind the norm of the dual space (C^)*, where is 
the domain of the generating family A[t,fi,u]. 
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5 Conclusion 



In this paper, our main aim is to analysis the sensitivity of the solution 
to a Hamilton-Jacobi-Bellman equation (j3.6p with respect to a functional 
parameter {fi.} € C{[0,T], Ai). This problem was first reduced to the sen- 
sitivity analysis with respect to a real- valued parameter a € [0,1]. Then 
we proved in Theorem 13.21 and Theorem 13.31 that the unique mild solution 
is Lipschitz continuous with respect to a. Finally, as an application of our 
sensitivity results, we gave verifiable conditions for the feedback regularity 
condition which is needed in mean field games model for solving the coupled 
backward- forward system (j4.5p - ()4.6p . 
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